The present invention is directed to a method and system which employ personal communications devices known in the art such as smartphones, personal digital assistant (PDA) type devices, tablet computers, and the like suitably equipped with built-in camera or other imaging capabilities. Personal communications devices are employed at multiple viewing positions for various collaborative applications to reliably acquire and/or track one or more common targets. The visual information obtained through images acquired by each device may be integrated with the device's inertial measurement information to augment the accuracy of positioning and/or tracking capabilities using the device.
The subject method and system improve upon various multiple sighting device systems and methods heretofore known, such as those disclosed in U.S. Pat. No. 7,225,548. In certain applications, the subject method and system enable multiple physically separated users to bring their image capture devices to bear on the same target. For example, the method and system enable various mobile personal communication devices to perform relative alignment functions using their internal cameras.
In accordance with certain aspects of the present invention, the subject system incorporates the smartphone, tablet, PDA, or other suitably equipped personal communications device of different users located within a proximate distance of one another. It may be desirable for one user to point out a vehicle, a bird in a tree, a person in a crowd, a sign, or a landscape feature of interest using the device without resorting to descriptive discussion. Each user may dynamically point features out to the other by suitable selection on the image displayed on his/her device. Similarly, a collection of users may all follow the lead of a first user. Such operations would be beneficial for use in various applications like military and law-enforcement, security, nature watching, scientific observation. They may also be beneficial, for example, in the context of augmenting social media to enable individuals to point out interesting things to their friends over a distance.
In other exemplary applications, the subject system may be used for time delayed finding of market targets in the 3D world. For example, a user, having established a virtual reference point in the physical word may record that information so that upon a revisit to the same general area they may spot the point and find it again easily. Accordingly, the system may be used to mark points of interest for oneself or for others. A camera view of the world on one's smartphone or tablet may in real-time combine available geo-location and orientation sensor data with the pixel level processing according to the process of information fusion described herein, to float visual markers or labels in the camera view as if they were attached to the physical world being observed. Thus, any user may view both the real world and a virtual overlay in a spatially convergent user interface. Such overlays have further potential in wearable computing solutions, such as electronic glasses, which obviate the need for the user hold up and point their computing device before them while walking.
Geo-tagged image or video transfer is an automatic function of many PDA type or other such personal communications devices, yet their corresponding tolerances tend to be wholly inadequate for meeting the needs of tactical teams or others requiring finer precision. For instance, GPS chipsets (when operational) provide accuracy at best on a scale of tens of meters, while compass modules (when present and sufficiently distant from interference) provide perhaps 5-10 degrees of orientation accuracy. In GPS-denied and metal-heavy military environments, accuracies suffer even more significantly.
Thus, built-in functions of currently existing mobile devices for personal communications are not sufficient to bring multiple cameras to bear on the same target with the accuracy and reliability in many applications, especially if a “slew-to-cue” capability or accurate tactical position details on an observed subject of interest are to be provided.
There is therefore a need for a method and system whereby personal communications devices at multiple imaging positions are coordinated to accurately and reliably bring their imaging fields of view upon a common target. There is a need, moreover, for such method and system to provide precise relative alignments between users' local points of view and provide reliable image-based device orientation information relative to the target for corrective integration with device IMU and related sensor package measurements. “IMU” is used contextually herein to describe any combination of sensors or sensor packages that may track orientation and changes in orientation, and, in certain embodiments, position and changes in position, including but not limited to accelerometers, angular accelerometers, gyroscopic measurements, compass heading, inclinometers, GPS, differential GPS, RF ranging, and so forth.